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ABSTRACT 

This paper describes and critiques a 1999 
employer/alumni study conducted by the University of Tennessee at 
Knoxville (UTK) and four other state postsecondary institutions. The 
study was designed to: (1) develop a questionnaire to tap opinion 
about graduates' preparation and a method of administration that 
could be used by both 2- and 4-year institutions of higher education; 
(2) determine employers 1 priorities for the content of a college 
education for their employees and to compare those priorities by 
institutional type and occupational classification; and (3) determine 
employers 1 perceptions of graduates 1 preparation and skills and to 
compare these perceptions with those expressed by graduates 
themselves, as well as to analyze these perceptions by institutional 
type and occupational classification. The study method involved: (1) 
mailing alumni surveys to all 1987-88 academic year graduates; and 
(2) sending a letter to each employed graduate explaining the study 1 s 
purpose and requesting permission to mail a questionnaire to his/her 
supervisor. Upon receiving permission, the employer questionnaire was 
sent to the supervisor with the graduate's signed permission 
statement. Results indicated that employers placed most value on oral 
and written communication skills, technical preparation, computing 
experience, and paid work experience in the field. College 
grade-point average was not related to employers' performance 
ratings. The critique notes, however, that the basic irsthodology has 
inherent problems with permission-granting rates for alumni low and 
employers' questionnaire responses highly positive, showing little 
variance. Telephone contact with alumni to obtain permission, coupled 
with interviews with employers who complete the mailed survey is 
recommended- Contains 34 references. (GLR) 
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CRITIQUE OF A METHOD FOR SURVEYING EMPLOYERS 

Five diverse public institutions in Tennessee used a state-wide survey of graduates 
two years out of college to identify alumni who would grant permission to survey their 
employers. The questionnaire developed for mailing to employers of graduates of both 
two- and four-year institutions drew a 91 percent return. Employers valued most oral and 
written communication skills, technical preparation, computing experience, and paid work 
in the field. College grade-point average was not related to employers' performance 
ratings. Telephone contact with graduates to obtain permission, coupled with follow-up 
interviews with a sample of employers who have completed a mailed survey, constitutes 
the most promising approach for future research. 
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CRITIQUE OF A METHOD FOR SURVEYING EMPLOYERS 

Colleges and universities are seeking ways to improve the quality of their programs 
and services and to demonstrate that quality to prospective students. Seymour (1991) 
asserts that institutions must have hard evidence of how they are doing in order to 
improve, and that measurement provides the feedback needed for improvement-the basis 
for a knowledge-driven organization. 

Most freshmen enter college to prepare for a career. Astin's annual Freshman 
Survey reveals that more than 75 percent of first-time freshmen say they are in college to 
"get a better job" and "to make more money" (Astin, Korn, & Berz, 1990). If this is the 
case, then it behooves institutions hoping to attract students to assess the job they are 
doing in preparing their graduates for employment and to pay attention to the findings. 
A survey program that periodically asks graduates and their employers about the job 
preparation the graduates have received from their home institution can play a vital role 
in providing the feedback for improvement that a quality-conscious institution demands 
from its comprehensive outcomes assessment program. 

In 1989, the University of Tennessee, Knoxville (UTK) was instrumental in bringing 
together six diverse public institutions in Tennessee to study the implications of the quality 
improvement methods of Edwards Deming (1986) and others for their outcomes 
assessment programs. Und«r the auspices of a grant from the Fund for the Improvement 
of Postsecondary Education (FIPSE), five of these institutions built upon an alumni survey 
program in which all were involved to develop a method for surveying the employers of 
graduates responding to the alumni survey. The purpose of this paper is to provide a 
description of this method and a critique of its use as a method for surveying employers. 
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Critique of Employer Survey 2 

Related Studies 

A review of the literature on the use of employer surveys in higher education reveals 
that most studies are unpublished and based on the experience of a single institution. 
Two-year institutions seek employer feedback more frequently than do four-year 
institutions, but the studies reported by both sectors are useful for the perspectives they 
provide on survey methodology and attendant response rates and analyses of employer 
responses. 

Survey methodology and response rates 

Two institutions report having used what might be called a "broadside" approach 
to surveying-sending questionnaires to employers only without identifying any particular 
graduates. A survey of community college employers in New York yielded a 47 percent 
response rate using placement records to obtain the names of the employers (Francis & 
Jones, 1976). St. Louis Community College at Forest Park identified employers from 
responses to an alumni survey (Kapraun & Nienkamp, 1985), a technique that attracted 
responses from 39 percent of the employers. 

In an attempt to obtain more specific information, many two-year colleges have 
linked their surveys of employers to named graduates. This approach necessitates 
obtaining the graduate's permission to question the employer, and the additional step 
lowers generalizability because those who grant permission may differ in important ways 
from those who do not. Institutions have employed various techniques to resolve these 
problems. Some have obtained permission as part of a mailed alumni survey, a technique 
yielding permission rates of 34 to 40 percent (Gell & Jones, 1975; Montemayor, Hardin, 
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& Reed, 1986; Head, 1990). Lorain County Community College attracted a 46 percent 
return by mailing postcards to alumni explaining the survey and asking for permission to 
contact the graduate's employer (Isbell & Jonas, 1976). Using telephone interviews with 
former students, Johnson County Community College obtained an 80 percent permission 
rate (Conklin, 1990), highest of all techniques described in the current literature. 

Regardless of the method used to obtain permission, the response rate on the part 
of employers has been high (Head, 1990; Case, 1986; Conklin. 1990; Maryland State 
Board for Community Colleges, 1986). Employers respond at the rate of 60 to 90 percent 
when asked for their opinions by institutions of higher education. Response rates are 
highest when the survey mailing includes a copy of the graduate's signed statement 
granting permission for the employer to respond. (Head, 1990). 

The literature suggests that four-year colleges and universities have much less 
experience with employer surveys than do two-year institutions, and that their techniques 
generally yield lower response rates. For instance, four-year institutions identifying 
employers through placement records have obtained response rates approximating 18 
percent (Knoblauch & German, 1989; Atkins & Kent, 1988). By seeking permission to 
contact employers through a telephone interview with alumni, Harper College obtained 
permission from 72 percent of its graduates and a response from 78 percent of their 
employers (Lucas, 1984). 
Survey Results 

Surveys of employers generally produce positive findings, regardless of the type of 
institution or the survey methodology applied. Employers rate most employee attributes 
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Critique of Employer Survey 4 

as important and most specified employees as good or very good. 

Technical, junior, and commumty colleges typically ask employers to assess the 
quality of their students' job preparation and job performance. Employers of two-year 
college graduates consistently report high satisfaction with both preparation and 
performance, and indicate that they would hire the graduate again if the opportunity arose 
(Gell & Jones, 1975; Kapraun & Nienkamp, 1985; Montemayor, Dominguez, & Reed, 1985; 
Isbell & Jonas, 1976; Case, 1986; Allred & Wingfield, 1982; O'Grady, 1990; Maryland State 
Board, 1986; Conklin, 1990; Francis & Jones, 1976). 

Employers give highest ratings to two-year graduates' technical skills; basic skills 
such as reading, writing, and arithmetic; and readiness for additional responsibilities 
(Maryland State Board, 1986; Kapraun & Nienkamp, 1985; Isbell & Jonas, 1976). Ability 
to communicate with supervisors, general communication skills, and ability to get along 
with fellow workers also receive high ratings (Gell & Jones, 1975; Isbell & Jonas, 1976; 
Kapraun & Nienkamp, 1985). The most highly rated traits or attitudes are general attitude 
toward work, willingness and ability to learn, acceptance of responsibility, personal 
initiative, ability to learn new tasks, and dependability (Gell & Jones, 1975; Montemayor 
et al, 1986; Kapraun & Nienkamp, 1985; Isbell & Jonas, 1976). 

Survey data collected at four-year institutions reveals strong interest in employers' 
opinions of the valuable attributes of a college education. According to employers, these 
attributes can be divided into three categories-areas of knowledge, skills, and attitudes or 
traits. The most valuable areas of knowledge are basic adult literacy, technical skills 
related to the job and the ability to apply one's knowledge (Mentkowski, O'Brien, 
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McEachern, & Fowler, 1982; Edge, 1985; Lindquist, 1985; Young, 1986; Cohen, 1984). 
Valuable skills identified by employers are those related to communication, ability to work 
with people, computing, organization/management, and research/investigation (Knoblauch 
& German, 1989; Lindquist, 1984; Young, 1986; Murphy & Jenks, 1983; Atkins & Kent, 
1988; Edge, 1985). Finally, the valuable traits or attitudes are dependability, attitudes 
toward work, trustworthiness, attitudes toward self and others, and self-motivation (Young, 
1986; Mentkowski et al. 1982; Edge, 1985; Murphy & Jenks, 1983). 

Pur poses of this Research 
This study was designed to serve three purposes: 1) To develop a questionnaire to 
tap opinion about graduates' preparation and a method of administration that could be 
used by both two-and four-year institutions of higher education; 2) to determine employers' 
priorities for the content of a college education for their employees and to compare those 
priorities by institutional type and occupational classification. 3) to determine employers' 
perceptions of graduates' preparation and skills and to compare these perceptions with 
some of those expressed by the graduates themselves, and at the same time, to analyze 
these perceptions by institutional type and occupational classification A limited discussion 
of the developmert of this survey and a more thoroughly developed critique of its 
performance follows. The survey method, response rates and findings are compared with 
corresponding information from previous studies. 

Met hods 

The employer survey for this study was developed by a consortium of five public 
institutions in Tennessee, including a technical institute, a community college, a 
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Critique of Employer Survey 6 

comprehensive university, a doctorate-granting institution, and the state's land-grant and 
research university. Because the Tennessee Higher Education Commission has required 
assessment of alumni opinion as part of its state-wide performance funding policy since 
1987, each of these institutions was using in alternate years a common survey for all 
alumni who had completed their studies two years previously (Banta, 1988). The five 
cooperating institutions decided to couple the employer survey with the current 
administration of the alumni survey in order to increase the amount of information that 
the study could include about the graduates whose employers responded to the employer 
survey. 

Questions for the survey instrument were derived by the authors from related 
literature ("Employer Input," 1990; Gardner, Kozlowski, & Broadus, 1988; Chamer, 1988; 
Eisenberg, Monge, and Farace, 1984; Phipps and Romesburg, 1988; Raza & Carpenter, 
1987) and were extensively reviewed by faculty and staff at all of the institutions in the 
consortium. Section One of the questionnaire asked about employers' perceptions of the 
relative value of the courses and related academic experiences typically offered on a college 
campus (see items in Table 1). Section Two listed employee traits and skills and asked 
employers to rate each in two ways, first in terms of its importance in the job performance 
of a person holding the position held by the graduate specified, and second in terms of that 
graduate's performance as compared "to others you have employed in similar positions" 
(see items in Table 2). Section Three asked employers to speculate about the satisfaction 
of the identified employee with his/her work and pay, then posed some global questions 
about the employee's performance and preparation. 

ERIC 1 0 



Critique of Employer Survey 7 

The items in the questionnaire for alumni were designed to serve the state's need 
to collect evidence of institutional accountability. They included the same questions about 
satisfaction with type of work and pay as were asked of employers. The graduates also 
were asked about the extent of their community participation and about their satisfaction 
with many aspects of their academic and social experiences in college. These responses 
were supplemented by cognitive achievement scores and additional demographic 
information derived from institutional student information systems. 

In the spring of 1990, each institution mailed alumni surveys to all its graduates of 
the 1987-88 academic year. One week after the initial mailing, a postcard reminder notice 
was mailed to non-respondents. A week after that, the questionnaire was sent again, with 
a new cover letter, to graduates who had not yet responded. Once the alumni survey 
response was obtained, each employed respondent was sent a letter explaining the purpose 
of the study of employer opinion and requesting permission to mail a questionnaire to 
his/her immediate supervisor. In addition, the graduate was asked to provide the 
supervisor's name and address, the title of the job in which he/she was employed, and a 
description of his/her work-related duties. The follow-up mailing to graduates was 
necessary to avoid violating the regulations governing the standardized method of 
administering the state-wide alumni survey. 

Upon receiving permission from the graduate, the employer questionnaire was sent 
to the graduate's supervisor with a copy of the graduate's signed permission statement. A 
postcard follow-up after one week and the full mailing after two weeks were used to 
encourage employer response. 
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TABLE 1 

1990 ALUMNI / EMPLOYER SURVEY 
MEAN RESPONSE TO VALUE ITEMS 
ALL INSTITUTIONS 


owe$Tton 


MEAN ... \ 
RESPONSE 


COURSES EMPHASIZING COMMUNICATIONS SKILLS 


2.89 


COURSES SPECIFIC TO THE AREA OF EMPLOYMENT 


2.79 


COMPUTER EXPERIENCE 


2 57 


SPECIFIC WORK EXPERIENCE IN FIELD 


2.64 


COURSES IN MATHEMATICS 


2.59 


GENERAL WORK EXPERIENCE 


2.58 


INTERNSHIP, CO-OP, OR FIELD EXPERIENCE 


2.57 


COURSES IN BUSINESS 


2.36 


COURSES IN STATISTICS 


2.32 


COURSES IN THE HUMANITIES 


2.29 


KNOWING HOW TO USE THE LIBRARY 


2.22 


HIGH GRADE POINT AVERAGE 


2.17 


COURSES IN SOCIAL SCIENCES 


2.14 


INVOLVEMENT IN A PROFESSIONAL ASSOCIATION 


2.11 


EXTRACURRICULAR ACTIVITIES 


1.89 


COURSES IN NATURAL SCIENCES 


1.87 


COURSES IN FINE ARTS 


1.51 


COURSES IN FORIEGN LANGUAGES 


1.38 
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row pros 

SPEAKING EFFECTIVELY 
WRITING EFFECTIVELY 
READING EFFECTIVELY 
LISTENING EFFECTIVELY 
DEFINING PROBLEMS 
SOLVING PROBLEMS 

UNDERSTANDING WRITTEN INFORMATION 

PROCESSING/INTERPRETING NUM. DATA 

WORKING WITH COMPUTERS 

APPLYING JOB RELATED TECHNICAL SKILLS 

THINKING CREATIVELY 

WORKING COOPERATIVELY 

LEADING OTHERS 

ADJUSTING TO NEW JOB DEMANDS 

BEING DEPENDABLE AND ON TIME 

WORKING UNDER PRESSURE 

WORKING WITH MINIMAL SUPERVISION 

HANDLING SEVERAL TASKS AT ONCE 

MAKING DECISIONS UNDER PRESSURE 

WORKING WITH PEOPLE FROM DIVERSE BKNDS. 



— 0341 " ' 



0.18414 
0.04127 
0.25326 

LiiSSi 

0.36461 
0.38489 
0.31582 
0.07941 
0.17759 
0.17914 
0.28589 
0-58329 
5£U3 
0.57679 

0-S1141 
0.76174 

0.73505 
0-68391 
0.66595 
0.58561 



ft 



IMPORTANCE ITEMS 


INTERPERSONAL 


TECHNICAL 


ftLAHMNtt PROJECTS — ' 


0.04*76 




SPEAKING EFFECTIVELY 


0.05482 


.0.01571 


WRITING EFFECTIVELY 


0.16236 


0.05706 


READING EFFECTIVELY 


0.03863 


0.03251 


LISTENING EFFECTIVELY 


•0.09025 


0.23161 


OEFINING PROBLEMS 


0.06377 


0.51209 


SOLVING PROBLEMS 


0.22942 


0.37034 


UNDERSTANDING WRITTEN INFORMATION 


0.21393 


0.22403 


PROCESSING/INTERPRETING NUM. DATA 


.0.05576 


0.70596 


WORKING WITH COMPUTERS 


0.01549 


9-60325 


APPLYING JOB RELATED TECHNICAL SKILLS 


0.00582 


p.646i3 


THINKING CREATIVELY 


0.29138 


0.40151 


WORKING COOPERATIVELY 


0.41437 


.0.06301 


LEADING OTHERS 


.0.4M2Z 


0.29597 


ADJUSTING TO NEW JOB DEMANDS 


9,52561 


0.09177 


BEING DEPENDABLE ANO ON TIME 


9r69573 


•0.04351 


WORKING UNDER PRESSURE 




0.12314 


WORKING WITH MINIMAL SUPERVISION 


0tM179 


•0.01181 


HANDLING SEVERAL TASKS AT ONCE 


Q.W956 


0.09948 


MAKING DECISIONS UNDER PRESSURE 


0,63291 


0.07869 


WORKING WITH PEOPLE FROM DIVERSE BKNDS. 


0,58181 


0.42392 



TABLE 2 

1990 ALUMNI / EMPLOYER SURVEY 
VARIMAX ROTATED FACTOR MATRICES 
IMPORTANCE FOR POSITION AND PERFORMANCE OF GRADUATES 

UTK ONLY 



•XI'IM-- 



0.05825 
0.30155 
0.22443 
0.09308 
G.47621 
0.51652 
0.43396 
0.70284 
0-65608 

0,58675 
•0.02831 
0.30703 
0.30896 
0.11908 
0.25238 
0.33925 
0.42416 
0.40359 
-0.00753 



BASIC SKILLS 
6.29926 

mm 

P.-.678S7 
0.72002 
9-35978 
0.30423 
0.13332 
0.27982 
•0.02522 
•0.10176 
•0.00731 
0.06791 
0.33335 
0.28939 
0.09141 
•0.11411 
•0.05182 
0.02251 
0.14845 
0.03202 
•0.03151 



mm 



0.72991 
0.74522 
0.66737 
0,5 74 39 
0,4,7326 
0.38215 
0.50347 
0.15636 
0.10368 
0.06951 
0.27218 
0.35552 
0.27887 
0.17568 
0.19079 
0.08938 
0.09829 
0.15069 
0.13954 
0.36928 
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Critique of Employer Survey 10 

Data on employers' opinions concerning the importance of various aspects of the 
graduate's education and performance were summarized for all participating institutions. 
In addition, all institutions used a common method of classifying the jobs of graduates so 
that cross-institutional analyses by occupational classification could be made (Lehr, 1991). 
However, only the data on graduates of the research university (UTK) were used to 
examine structural characteristics of the survey because more background data on 
graduates could be obtained at that institution than at the others. 
Characteristics of Respondents 

The rate of return for the population of 1988 graduates was 53 percent at UTK, and 
between 21 and 50 percent at the other institutions. Seventy-eight percent of the UTK 
respondents were employed, and 21 percent of these employed graduates granted 
permission to contact their employers. The average response rate for employers of all the 
institutions' graduates was 91 percent. These response rates were within the ranges 
reported in previous studies. 

UTK graduates who responded to the alumni survey had slightly higher entering 
ACT scores and grade-point averages than non-respondents, but the differences did not 
meet a test of practical significance that is applied throughout this study, that is, there is 
a difference of more than five percent between the statistics in question. Graduates giving 
permission for their supervisor to receive a questionnaire did not differ on any known 
demographic variables from those who did not. Those giving permission were slightly 
more satisfied with their work, but, as Head (1990) also concluded, the difference was not 
practically significant. 

erJc 1 4 
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Employer Responses 

Consistent with the findings reported in the literature, employers' responses across 
the institutions making up the consortium were strong and positive. Their ratings 
produced little variation among items in the scales of the questionnaire. 

Valuable aspects of a college education . Asked about aspects of a college education 
that were of value to a person holding the job of the graduate identified for them, 
employers rated all but 4 of 18 items somewhat to very valuable, that is, between 2 and 
3 on a 3-point scale (see Table 1). Courses emphasizing oral and written communications 
skills were deemed most important, followed by courses specific to the area of employment, 
computing experience, and paid work experience in the field. Courses in foreign languages, 
fine arts, and natural sciences, and extracurricular activities, were the four items receiving 
mean ratings below 2 on the 3-point scale. 

Differences were found between two-year and four-year institutions in terms of 
employers' responses to just three of the items bearing on value of college experiences. 
Employers of four-year graduates considered extracurricular activities and courses in 
foreign languages and fine arts to be of more value to successful job performance of the 
graduates they had hired than did employers of two-year graduates. On more than half 
of the items describing valuable college experiences, responses differed by occupation of 
employee. Courses in business were valued most by employers of graduates in sales, 
administrative, and clerical fields. Employers of writers and artists and "other 
professionals" saw more value than did employers of other occupational groups in knowing 
how to use the library, engaging in an internship or field experience, and taking courses 
in foreign languages, fine arts, humanities, and courses specific to the area of employment. 

9 
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Important employee traits . In terms of importance in the successful job performance 
of a person holding the position of the identified graduate, all but one of 21 employee 
attributes and behaviors were rated 2.5 or higher on a 3-point scale. Leading the list at 
2.94 was "listening effectively," last was "leading others." No differences were found 
between employers of two-year and four-year graduates in terms of the importance they 
attach to worker traits and skills. However, there were differences on almost half of these 
items on the basis of occupational category. Managers are perceived to need the highest 
levels of the most diverse array of skills; on six of ten items, employers' ratings of the 
importance for managers is highest or second-highest among the eight occupational 
categories. Employers of graduates working in sales deem it most important that these 
workers be able to speak effectively and make decisions under pressure. Employers 
apparently associate word processing with the skills of "working with computers" and 
"processing...data" since their ratings indicate that these skills are most important for 
workers in clerical occupations. 

Employee performance . Every rating of employee performance on the 21 -item scale 
in Section Two was above 2.8, that is, in the range of good to excellent on a 4-point scale. 
Employers of 4-year graduates view their employees as more competent at complex and 
integrative skills than do employers of two-year graduates. Just two items showed 
differences by occupational category-clerical workers perform most competently in dealing 
with computers, and managers are best at leading others. 

Satisfaction with work and p ay. Graduates employed in their fields are more 
satisfied with their jobs than are those not so employed. There is a low but significant 
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correlation between graduates' levels of satisfaction with their work and with their pay and 
employers' perceptions of these levels of satisfaction. 

Employers' global assessments of graduates as employees . Employers of the 
graduates of the research university were more likely to say they would hire the identified 
graduate again and promote the graduate; however, there was no clear advantage for four- 
year institutions on these two global items. Nevertheless, employers of four-year graduates 
did perceive that these employees possessed more of the characteristics they expected of 
college graduates than did employers of two-year graduates. Perhaps as important as the 
significant differences that were found were those that were not significant: there were 
no differences among institutions in terms of employers' global ratings of job performance, 
quality of general education preparation, or education in the area of specialization. In 
addition, there were no differences on any of the global items in terms of employer 
satisfaction with the performance of graduates in different occupational categories. 
Structural Analysis of the Employer Questionnaire 

Section One questions about the value of various educational experiences for a 
person holding a particular job were of only marginal utility. Employers showed very little 
inclination to rate any college experience as less than important for most positions. 
However, factor analysis of employer responses in Section Two provides evidence that 
employers do think in terms of three sets of skills-basic, technical, and interpersonal-when 
they think of positions and the importance of various skills to successful job performance 
(see Table 2). Basic skills are those of reading, writing, and speaking effectively. 
Technical skills have to do with problem-solving and working with numbers and computers. 
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Interpersonal skills include the ability to work cooperatively and to lead as well as to 
handle multiple tasks with minimal supervision. The same dimensional structure can be 
developed for the importance ratings (the degree to which a skill is seen as important for 
a job) and the performance ratings (the evaluation of graduates' performance by their 
supervisors on the same items). 

The unique characteristics of the data collected are revealed in a model of the 
evaluation process (see Figure 1). The overall employer performance rating is cast as the 
dependent variable, and path analysis is used to estimate the effects of independent 
variables representing the entering ACT score, high school and college grade-point 
averages, and mother's education of responding graduates, as well as their performance 
scores on scales constructed from variables loading on the three performance dimensions 
described earlier. Forty-two percent of the variance in overall employee rating is explained 
in the resulting model. The entering ACT score is almost as good at predicting 
performance ratings as is the employer's judgment of basic skills performance of the 
graduate on the job, and the variation among graduates in terms of college grade point 
average is remarkably unrelated to any performance measure. 

Limitations of the Design and Methods of Implementation 

Since all 1987-88 graduates at the institutions participating in this study were 
scheduled to receive a mailed questionnaire in 1990, the consortium representatives made 
the pragmatic decision to use this alumni survey--an instrument with a history of rather 
successful use in the state-as the basis for its proposed employer survey. Thus it was not 
possible to include the request for permission to contact the employer in the initial mailing 
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Critique of Employer Survey 16 

to graduates. Telephone contact with graduates for the purpose of obtaining permission, 
a proven technique for increasing response rates, was ruled out primarily due to its cost. 
Moreover, there is within the consortium a long-standing tradition of using mail surveys, 
and the principals in the study felt that more information could be collected more 
systematically by mail than by telephone. Quantity and quality of the resulting data were 
also factors in the decision to use a predetermined set of questions for employers as 
opposed to an open-ended format. Unfortunately, the literature contains evidence that 
employers provide almost uniformly high positive responses when the response categories 
are established for them. 

While the rate of return from employers in this study was relatively high, the total 
number of employe" questionnaires available for analysis was small because so few 
graduates gave their permission for contact to be made with their employers. Although 
alumni granting permission did not differ on important demographic characteristics from 
their classmates who did not return the permission form, the group giving permission did 
appear to be more gregarious and involved in their careers, as evidenced in their contacts 
with faculty while in college and in their greater rates of participation after college in job- 
related seminars and professional organizations. 
Strengths of the Design and Methods of Implementation 

FIPSE funJing made it possible for institutions separated by as much as 500 miles 
to plan a common survey methodology suitable for use at both two-year and four-year 
institutions and thus capable of generating comparative data based on institutional type 
that had not been available for analysis heretofore. FIPSE funds also supported the 
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development of a clear and relatively simple method of using the Dictionary of 
Occupational Titles (U.S. Employment Service, 1977) to classify occupations that can be 
applied in other surveys. The alumni survey, coupled with data from student records, 
provided a wealth of information about graduates' college experiences. Having these data 
made it possible to construct a model of factors contributing to employers' assessment of 
employee performance. 

Finally, there were several pieces of evidence that the employer survey developed 
for use in this study is technically sound. Employers returned it at a rate exceeding 90 
percent, and even though given the opportunity to write comments, they made no negative 
comments about the questionnaire and few marks on the instrument itself that indicated 
frustration with the response alternatives. Additionally, employers' ratings of the 
importance of specified educational experiences and employee attributes were related in 
expected ways to occupational classification. 

The factor analysis and subsequent path analysis used to construct the model of 
employer evaluation of overall employee performance demonstrated the utility of the multi- 
phase, multi-level approach used in this research. The factor analyses strongly suggest that 
employers think in terms of three dimensions-basic, technical, and interpersonal skills--as 
they evaluate employee performance, and that basic and interpersonal skills are more 
important factors in this evaluation than are specific job-related technical skills. The other 
background characteristics make it possible to explore the varying levels of influence that 
such experiences and characteristics have on development of the skills employers value. 
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Conclusion 

This study produced an employer survey capable of serving the interests of two- and 
four-year institutions and of attracting a high rate of return on the part of employers. The 
methodology also incorporated an alumni questionnaire that contributed important 
information about the background characteristics and college experiences of the graduates 
who responded. However, the percentage of alumni giving permission for subsequent 
contact with their employers was disappointingly low. 

The findings of the initial phase of this research raise questions about the utility of 
a mail survey for employers that uses predetermined questions and a multiple-choice 
response mode. Employers' responses are so positive that they appear to be systematically 
biased in some way. The Hawthorne effect may be operating; that is, employers are so 
pleased to be invited to share their opinions about employees with colleges and universities 
that they tend to respond positively. Perhaps graduates give permission only when they 
feel their employer is the type of person who will say positive things about them. Yet 
another alternative is that, in fact, the positive ratings reflect reality: Most employees have 
been selected carefully and trained on the job, and they remain employed because they are 
doing satisfactory work, thus satisfying their employers. Further study of this question is 
clearly needed. 

The authors' experience to date indicates that no matter how good the 
questionnaire, mailing a request for permission to alumni, then mailing a multiple-choice 
quastionnaire to employers, is a methodology with inherent problems. Permission-granting 
rates for alumni will be low, and employers' questionnaire responses will be highly positive, 
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showing little variance. Telephone contact with graduates to obtain permission, coupled 
with face-to-face follow-up interviews with a sample of employers who have completed the 
mailed survey, appears to be a more promising method for future research. In addition, 
items on the questionnaire for employers could be constructed to force respondents to 
distribute a given number of points over a group of response categories, thus compelling 
them to make decisions about the relative worth of the options. 

Finally, if an important goal of higher education is to prepare graduates fo r 
employment, employers' ssponses provide strong evidence that faculty should focus their 
efforts particularly upon strengthening the basic skills and interpersonal skills of their 
students. Specialized skills imparted by education in the major field are of secondary 
importance when compared with these fundamental competences. This finding also has 
implications for the assessment of outcomes in higher education: Future efforts should be 
aimed at developing better measures of students' basic skills and interpersonal skills. In 
addition, developers of comprehensive campus assessment programs should be strongly 
encouraged to include these measures in their schedules of activities. 
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